Display control system and display control method for changing a size of an object image in association with a location change of an operation device

ABSTRACT

A display control system according to the present disclosure includes a cross-section display which, when an operation device moves in a direction away from a display device and overlaps a virtual plane at a position separated from the display device by a predetermined distance, causes the display device to display the object image including a cut plane obtained by cutting the operation device at the virtual plane.

INCORPORATION BY REFERENCE

This application is based upon and claims the benefit of priority fromthe corresponding Japanese Patent Applications No. 2020-068984 filed onApr. 7, 2020, and No. 2020-068985 filed on Apr. 7, 2020, the entirecontents of which are incorporated herein by reference.

The present disclosure relates to a display control system and a displaycontrol method.

BACKGROUND

Conventionally, there has been proposed a system in which a computergraphics (CG) image of an object to be viewed is presented on a displaydevice in accordance with the movement of a replica (model, doll) thatimitates the shape of the object to be viewed created as CG data.According to this system, it is possible to present an image viewed fromany direction that the viewer desires to see on the display device.

However, although the conventional system can change the orientation ofthe CG image on the display device in association with the movement ofthe replica, it is difficult to present detailed information(cross-section information, description, explanation, and the like)relating to the object to be viewed to the viewer in association withthe movement of the replica. In particular, when the object to be viewedis an art object or the like, it is desirable that detailed informationrelating to the art object can be presented to the viewer.

SUMMARY

An object of the present disclosure is to provide a display controlsystem which changes the orientation of an object image of an operationobject displayed on a display device in association with the orientationchange of an operation device, and which can display detailedinformation of the operation object, and a display control methodthereof.

A display control system according to an aspect of the presentdisclosure changes an orientation of an object image of an operationobject displayed on a display device in association with an orientationchange of an operation device, and includes a cross-section displaywhich, when the operation device moves in a direction away from thedisplay device and overlaps a virtual plane at a position separated fromthe display device by a predetermined distance, causes the displaydevice to display the object image comprising a cut plane obtained bycutting the operation device at the virtual plane.

Further, a display control system according to an aspect of the presentdisclosure changes an orientation of an object image of an operationobject displayed on a display device in association with an orientationchange of an operation device, and incudes a marker detector thatdetects a marker set at a specific position on the object image, and aninformation presenter that, when the marker is detected by the markerdetector, presents specific information associated with the detectedmarker.

A display control method according to another aspect of the presentdisclosure changes an orientation of an object image of an operationobject displayed on a display device in association with an orientationchange of an operation device, and when the operation device moves in adirection away from the display device and overlaps a virtual plane at aposition separated from the display device by a predetermined distance,executes by one or more processors to cause the display device todisplay the object image including a cut plane obtained by cutting theoperation device at the virtual plane.

Further, a display control method according to another embodiment of thepresent disclosure changes an orientation of an object image of anoperation object displayed on a display device in association with anorientation change of an operation device, detects a marker set at aspecific position of the object image, and presents, when the marker isdetected, specific information associated with the detected marker,wherein detecting the marker and presenting the specific information areexecuted by one or more processors.

A recording medium according to another aspect of the present disclosurestores a program executed by one or more processors to, in a displaycontrol method that changes an orientation of an object image of anoperation object displayed on a display device in association with anorientation change of an operation device, when the operation devicemoves in a direction away from the display device and overlaps a virtualplane at a position separated from the display device by a predetermineddistance, cause the display device to display the object image includinga cut plane obtained by cutting the operation device at the virtualplane.

Further, a recording medium according to another embodiment of thepresent disclosure stores a program executed by one or more processorsto, in a display control method that changes an orientation of an objectimage of an operation object displayed on a display device inassociation with an orientation change of an operation device, detect amarker set at a specific position of the object image, and present, whenthe marker is detected, specific information associated with thedetected marker.

According to the present disclosure, in a display control system thatchanges the orientation of an object image of an operation objectdisplayed on a display device in association with the orientation changeof an operation device, it is possible to display detailed informationof the operation object.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription with reference where appropriate to the accompanyingdrawings. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter. Furthermore,the claimed subject matter is not limited to implementations that solveany or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration of a displaycontrol system according to an embodiment of the present disclosure.

FIG. 2 is a block diagram illustrating the configuration of the displaycontrol system according to the embodiment of the present disclosure.

FIG. 3 is a view illustrating an appearance of an operation deviceaccording to the embodiment of the present disclosure.

FIG. 4A is a view illustrating an example of a method for setting afacing orientation of the operation device according to an embodiment ofthe present disclosure.

FIG. 4B is a view illustrating an example of the method for setting thefacing orientation of the operation device according to the embodimentof the present disclosure.

FIG. 4C is a view illustrating an example of the method for setting thefacing orientation of the operation device according to the embodimentof the present disclosure.

FIG. 5 is a view illustrating an example of an object image displayed ona display device according to the embodiment of the present disclosure.

FIG. 6 is a view illustrating an example of the object image displayedon the display device according to the embodiment of the presentdisclosure.

FIG. 7 is a view illustrating an example of a marker of the object imagedisplayed on the display device according to the embodiment of thepresent disclosure.

FIG. 8 is a view illustrating an example of the object image displayedon the display device according to the embodiment of the presentdisclosure.

FIG. 9 is a view illustrating an example of the object image displayedon the display device according to the embodiment of the presentdisclosure.

FIG. 10 is a view for explaining a cross-section display mode in thedisplay control system according to the embodiment of the presentdisclosure.

FIG. 11 is a view illustrating an example of the object image displayedon the display device according to the embodiment of the presentdisclosure.

FIG. 12 is a flowchart illustrating an example of a procedure of displaycontrol processing executed by the operation device according to theembodiment of the present disclosure.

FIG. 13 is a view illustrating a relative position between the operationdevice and the user in the display control system according to theembodiment of the present disclosure.

FIG. 14 is a view illustrating an appearance of the operation deviceaccording to the embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will now be described withreference to the accompanying drawings for the purpose of understandingthe present disclosure. It should be noted that the followingembodiments are examples that embody the present disclosure, and do notlimit the technical scope of the present disclosure.

Display Control System 100

As illustrated in FIGS. 1 and 2 , a display control system 100 accordingto an embodiment of the present disclosure includes an operation device1 and a display device 2. The operation device 1 and the display device2 can communicate with each other via a communication network N1 such asa wireless local area network (LAN) or a wired LAN. The operation device1 is an example of an operation device of the present disclosure, andthe display device 2 is an example of a display device of the presentdisclosure.

The display control system 100 can change the orientation (posture orattitude) of an object image 3 of an operation object displayed on thedisplay device 2 in association with the orientation change of theoperation device 1 having a shape simulating the operation object. Forexample, the operation object is an art object, the object image 3 is athree-dimensional image of the art object, and the operation device 1 isa replica (model) of the art object. In the present embodiment, apottery tea bowl will be described as an example of the art object. Theoperation device 1 is, for example, a replica having the same material,shape, size, weight, texture, or the like as the tea bowl.

For example, as illustrated in FIG. 1 , when a user (viewer) holds androtates the operation device 1 (replica of the tea bowl) in the D1direction, the display control system 100 rotates the object image 3displayed on the display device 2 in the D1 direction according to theuser's operation. Further, for example, when the user holds and rotatesthe operation device 1 in the D2 direction, the display control system100 rotates the object image 3 in the D2 direction according to theuser's operation. Further, for example, when the user extends her or hisarm while holding the operation device 1 to move the operation device 1in a direction away from her or his face (rear side), that is, when theuser brings the operation device 1 closer to the display device 2 side,the display control system 100 displays the object image 3 with areduced size on the display device 2. Furthermore, for example, when theuser folds her or his arm while holding the operation device 1 to movethe operation device 1 in a direction closer to her or his face (frontside), that is, when the user moves the operation device 1 away from thedisplay device 2, the display control system 100 displays the objectimage 3 with an enlarged size on the display device 2.

As described above, the operation device 1 is a controller capable ofchanging the orientation of the object image 3 displayed on the displaydevice 2 according to the operation of the user. The art object (forexample, a tea bowl) is an example of an operation object according tothe present disclosure. Further, the object image 3 is an example of anobject image of the present disclosure.

In the present embodiment, the display control system 100 corresponds toa display control system according to the present disclosure, but thedisplay control system according to the present disclosure may berealized by the operation device 1 alone or the display device 2 alone.

Operation Device 1

As illustrated in FIG. 2 , the operation device 1 includes a controller11, a storage 12, an orientation detection sensor 13, a communicator 14,and the like. Various processing executed by the operation device 1 maybe executed by one or more processors in a distributed manner. FIG. 3illustrates an external view of the operation device 1. In the presentembodiment, the operation device 1 is a replica of a tea bowl.

The communicator 14 is a communication interface for connecting theoperation device 1 to the communication network N1 by wire or wirelesslyand executing data communication according to a predeterminedcommunication protocol with an external device such as the displaydevice 2 via the communication network N1.

The orientation detection sensor 13 is a sensor for detecting theorientation of the operation device 1, and includes a gyro sensor, anacceleration sensor, a geomagnetic sensor, and the like. For example,the orientation detection sensor 13 detects an orientation change suchas the rotation or inclination of the operation device 1, and outputs adetection signal to the controller 11. The orientation detection sensor13 is mounted on the operation device 1. In the present embodiment, forexample, as illustrated in FIG. 3 , the orientation detection sensor 13is fixed to the bottom portion inside the tea bowl. Note that theorientation detection sensor 13 may be covered with a lid so as not tobe seen from the outside, or may be built in the operation device 1.

The storage 12 is a non-volatile storage such as a hard disk drive(HDD), a solid state drive (SSD) or a flash memory for storing variouskinds of information. The storage 12 stores a control program such as adisplay control program for causing the controller 11 to execute displaycontrol processing (see FIG. 12 ) to be described later. For example,the display control program is non-temporarily recorded on acomputer-readable recording medium such as a universal serial bus (USB),a compact disc (CD) or a digital versatile disc (DVD). Then, the displaycontrol program is read by a reading device (not illustrated) such as aUSB drive, a CD drive or a DVD drive electrically connected to theoperation device 1, and then is stored in the storage 12. Further, thedisplay control program may be downloaded from a server accessible fromthe operation device 1, and then stored in the storage 12.

The controller 11 includes control devices such as a central processorunit (CPU), a read only memory (ROM) and a random access memory (RAM).The CPU is a processor for executing various kinds of arithmeticprocessing. The ROM is a non-volatile storage in which control programssuch as a basic input output system (BIOS) and an operation system (OS)for causing the CPU to execute various arithmetic operations are storedin advance. The RAM is a volatile or non-volatile storage for storingvarious kinds of information, and is used as a temporary storage memory(work area) for various kinds of processing executed by the CPU. Then,the controller 11 controls the operation device 1 by executing variouscontrol programs stored in advance in the ROM or the storage 12 by theCPU.

Specifically, as illustrated in FIG. 2 , the controller 11 includesvarious types of processors such as an orientation setter 111 and anorientation detector 112. Note that the controller 11 functions as theorientation setter 111 and the orientation detector 112 by executingvarious processing according to the display control program by the CPU.Further, a part or all of the processors in the controller 11 may beconfigured by an electronic circuit. Note that the display controlprogram may be a program for causing a plurality of processors tofunction as the various types of processors.

The orientation setter 111 sets the orientation of the operation device1 to the facing orientation in which the operation device 1 faces theuser. Specifically, the orientation setter 111 registers the orientation(facing orientation) of the operation device 1 at the position where theuser faces the operation device 1, that is, at the position where theuser places at the front of the operation device 1. For example, theorientation setter 111 acquires and registers the coordinates and theinclination of the operation device 1 in the X, Y and Z directions inthe facing orientation from the orientation detection sensor 13.

For example, the orientation setter 111 may use a geomagnetic sensor(compass) provided on the operation device 1 to set the facingorientation. For example, as illustrated in FIG. 4A, the display device2 is positioned in advance so as to face a predetermined direction A,and the operation device 1 is placed at the front of the display device2 for a predetermined time (for example, five seconds). Then, theorientation setter 111 calculates the front direction of the operationdevice 1 with respect to the display device 2 based on the relativerelationship between the direction A of the display device 2 and adirection B (N pole) of the geomagnetic sensor. As a result, theorientation setter 111 sets the orientation of the operation device 1 tothe facing orientation.

Further, for example, the orientation setter 111 may use a geomagneticsensor (compass) provided on the operation device 1 and a magnet Mgdisposed in front of the display device 2 to set the facing orientation.For example, as illustrated in FIG. 4B, the magnet Mg is arranged infront of the display device 2, and the operation device 1 is placed infront of the magnet Mg for a predetermined time (for example, fiveseconds). Then, the geomagnetic sensor detects the direction of themagnet Mg, and the orientation setter 111 calculates the front directionof the operation device 1 with respect to the display device 2 based onthe direction detected by the geomagnetic sensor. As a result, theorientation setter 111 sets the orientation of the operation device 1 tothe facing orientation.

Further, for example, the orientation setter 111 may use an accelerationsensor provided on the operation device 1 to set the facing orientation.For example, as illustrated in FIG. 4C, an inclined table T, which isinclined by an angle d (for example, five degrees) toward the displaydevice 2, is installed in front of the display device 2, and theoperation device 1 is placed on the inclined table T for a predeterminedtime (for example, five seconds). Note that the dotted line in FIG. 4Crepresents the position of the horizontal table. Then, the accelerationsensor detects acceleration in the X-axis, Y-axis, and Z-axis, and theorientation setter 111 calculates the front direction of the operationdevice 1 with respect to the display device 2 based on each accelerationdetected by the acceleration sensor. As a result, the orientation setter111 sets the orientation of the operation device 1 to the facingorientation.

Further, for example, the orientation setter 111 may set the orientationof the operation device 1 when the user presses a calibration button(not illustrated) provided on the operation device 1 to the facingorientation. For example, the user grips and positions the operationdevice 1 so that the operation device 1 faces the front direction of theuser. Note that the terms “hold” and “grip” are interchangeable.Thereafter, when the user presses the calibration button, theorientation setter 111 sets the orientation of the operation device 1 atthat time to the facing orientation. Note that the function of thecalibration button may be replaced with a predetermined operation to theoperation device 1. For example, when the user positions the operationdevice 1 and then shakes the operation device 1 in a predetermineddirection, the orientation setter 111 sets the orientation of theoperation device 1 at that time to the facing orientation.

Further, for example, the orientation setter 111 may set the facingorientation by using a camera (not illustrated) provided in theoperation device 1. For example, the camera images the user and thesurrounding environment such as the ceiling, floor and wall around theuser, and the orientation setter 111 calculates the front direction ofthe operation device 1 with respect to the display device 2 based on thecaptured image acquired from the camera. As a result, the orientationsetter 111 sets the orientation of the operation device 1 to the facingorientation.

As described above, the orientation setter 111 can set the frontdirection (facing orientation) of the operation device 1 by variousmethods. Further, the orientation setter 111 can set the facingorientation every time the user places the operation device 1 at apredetermined position. Note that when the front, back or otherorientation is set to the operation device 1 itself, the orientationsetter 111 sets the facing orientation in consideration of theorientation of the operation device 1. For example, when the operationdevice 1 is placed at a predetermined position so as to face the userside (the front side of the display device 2), the orientation setter111 sets the facing orientation. As a result, the display device 2 candisplay the object image 3 so that the object image 3 faces the userside (the front side of the display device 2). Further, the orientationsetter 111 may detect the orientation of the operation device 1 placedat a predetermined position, and set the facing orientation. In thiscase, the display device 2 displays the object image 3 so that theorientation of the object image 3 is the same as the orientation of theoperation device 1.

When the user changes the orientation of the operation device 1, theorientation detector 112 detects the orientation change of the operationdevice 1. Specifically, the orientation detector 112 detects theorientation change of the operation device 1 based on the detectionsignal acquired from the orientation detection sensor 13. Theorientation detector 112 is an example of an orientation detector of thepresent disclosure.

For example, as illustrated in FIG. 1 , when the user desires to changethe orientation of the object image 3 displayed on the display device 2,the user holds and rotates the operation device 1 in the right directionD1 or the left direction D2. In this case, the orientation detector 112detects the orientation change (right rotation or left rotation) of theoperation device 1 based on the detection signal acquired from theorientation detection sensor 13.

Further, for example, as illustrated in FIG. 1 , when the user desiresto change the size (display magnification) of the object image 3displayed on the display device 2 (for example, desires to enlarge theobject image 3), the user holds and moves the operation device 1 to thefront side. In this case, the orientation detector 112 detects theorientation change (movement toward the front side) of the operationdevice 1 based on the detection signal acquired from the orientationdetection sensor 13.

When the orientation detector 112 detects the orientation change of theoperation device 1, the orientation detector 112 outputs information(orientation information) corresponding to the orientation change fromthe facing orientation to the display device 2. The orientationinformation includes information such as a rotation angle, a tilt angle,a coordinate, and a display magnification (enlargement ratio, reductionratio).

Display Device 2

As illustrated in FIG. 2 , the display device 2 includes a controller21, a storage 22, a display 23, a position detection sensor 24, acommunicator 25, and the like. The display device 2 may be aninformation processing device such as a personal computer, for example.

The communicator 25 is a communication interface for connecting thedisplay device 2 to the communication network N1 by wire or wirelesslyand executing data communication according to a predeterminedcommunication protocol with an external device such as the operationdevice 1 through the communication network N1.

The position detection sensor 24 detects the position of the operationdevice 1 with respect to the display device 2. Specifically, theposition detection sensor 24 detects the position (X, Y and Zcoordinates) of the operation device 1 with the installation location ofthe position detection sensor 24 as a reference (coordinate origin), forexample. For example, the position detection sensor 24 detects thedistance and position to the operation device 1 by irradiating infraredrays to the operation device 1 and detecting the reflected light fromthe operation device 1. Note that the position detection sensor 24 maybe a camera. In this case, the position detection sensor 24 detects thedistance and position to the operation device 1 based on the imagecaptured by the camera. The position detection sensor 24 may be providedoutside the display device 2. For example, the position detection sensor24 may be provided on the ceiling or wall of the room where theoperation device 1 and the display device 2 are installed.

The display 23 is a liquid crystal display or an organic electroluminescence (EL) display for displaying various kinds of information.The display 23 displays, for example, the object image 3. Note that thedisplay device 2 may include an operator (not illustrated) such as amouse, a keyboard, or a touch panel for receiving various operations.

The storage 22 is a non-volatile storage such as an HDD, an SSD, or aflash memory for storing various kinds of information. Data(three-dimensional image data) of the object image 3 displayed on thedisplay 23 is stored in the storage 22.

Further, a control program such as a display control program for causingthe controller 21 to execute display control processing (see FIG. 12 )to be described later is stored in the storage 22. For example, thedisplay control program is non-temporarily recorded on acomputer-readable recording medium such as an USB, a CD, or a DVD. Then,the display control program is read by a reading device (notillustrated) such as a USB drive, a CD drive or a DVD drive electricallyconnected to the display device 2, and is then stored in the storage 22.Further, the display control program may be downloaded from a serveraccessible from the display device 2 and then stored in the storage 22.

The controller 21 includes control devices such as a CPU, a ROM, and aRAM. The CPU is a processor for executing various kinds of arithmeticprocessing. The ROM is a non-volatile storage in which control programssuch as a BIOS and an OS for causing the CPU to execute variousprocessing are stored in advance. The RAM is a volatile or non-volatilestorage for storing various kinds of information, and is used as atemporary storage memory (work area) for various kinds of processingexecuted by the CPU. Then, the controller 21 controls the display device2 by executing various control programs stored in the ROM or the storage22 in advance by the CPU.

Specifically, as illustrated in FIG. 2 , the controller 21 includesvarious types of processors such as a display processor 211, anorientation controller 212, a marker detector 213, an informationpresenter 214, and a cross-section display 215. Note that the controller21 functions as the display processor 211, the orientation controller212, the marker detector 213, the information presenter 214, and thecross-section display 215 by executing various processing according tothe display control program by the CPU. Further, a part or all of theprocessors in the controller 21 may be configured by an electroniccircuit. Note that the display control program may be a program forcausing a plurality of processors to function as the various types ofprocessors.

The display processor 211 causes the display 23 to display various kindsof information. For example, the display processor 211 causes thedisplay 23 to display the object image 3 (see FIG. 1 ).

The orientation controller 212 receives the orientation information fromthe operation device 1, and executes processing corresponding to theorientation information on the object image 3 displayed on the display23. Specifically, the orientation controller 212 changes the orientationof the object image 3 in accordance with the orientation change of theoperation device 1. The orientation controller 212 changes theorientation of the object image 3, based on the rotation angle of theoperation device 1 corresponding to the orientation change of theoperation device 1 and the position of the operation device 1 withrespect to the display device 2. The orientation controller 212 is anexample of an orientation controller of the present disclosure.

For example, when the user moves the operation device 1, the orientationdetector 112 of the operation device 1 outputs orientation informationincluding information such as a rotation angle, a tilt angle,coordinates, and a display magnification acquired from the orientationdetection sensor 13 to the display device 2. When the orientationcontroller 212 receives the orientation information, the orientationcontroller 212 changes the rotation angle, the tilt angle, thecoordinates, the display magnification, and the like of the object image3 based on the orientation information. Note that when the front, backor other orientation is set to the operation device 1 itself, theorientation controller 212 changes the rotation angle, the tilt angle,the coordinates, the display magnification, and the like of the objectimage 3 in a state where the orientation of the operation device 1matches the orientation of the object image 3. Further, the orientationcontroller 212 may calculate the display coordinates and the displaymagnification of the object image 3 based on the detection signal fromthe position detection sensor 24.

For example, as illustrated in FIG. 5 , when the user moves theoperation device 1 to the front side so as to bring the operation device1 closer to her or his face, the orientation controller 212 calculatesthe enlargement ratio corresponding to the movement amount, and changes(enlarges) the display magnification of the object image 3. Further, forexample, as illustrated in FIG. 6 , when the user moves the operationdevice 1 to the rear side so as to move the operation device 1 away fromher or his face, the orientation controller 212 calculates the reductionratio corresponding to the movement amount, and changes (reduces) thedisplay magnification of the object image 3.

The orientation controller 212 controls the orientation (display state)of the object image 3 displayed on the display 23 so that theorientation of the object image 3 matches the orientation of theoperation device 1 visible to the user. Therefore, for example, when theuser turns the bottom of the tea bowl (operation device 1) toward theuser herself or himself, the orientation controller 212 rotates the teabowl of the object image 3 so that the bottom of the tea bowl faces theuser.

Note that the orientation controller 212 may also have the function ofthe orientation detector 112 of the operation device 1. In this case,the orientation detector 112 may be omitted from the operation device 1.

The marker detector 213 detects a marker M set at a specific position ofthe object image 3. For example, as illustrated in FIG. 7 , a pluralityof markers M1, M2 and M3 are set on the object image 3 at eachpredetermined specific position. The number of markers M is notnecessarily limited to plural, and may be one. For example, an exhibitorwho exhibits a tea bowl at an exhibition sets a marker M at acharacteristic portion of the tea bowl. Each marker M is displayed onthe display device 2 so as to be distinguishable from the object image3. For example, the marker M is displayed by lighting or blinking on thedisplay device 2.

Specifically, the marker detector 213 detects the marker M when themarker M faces the front direction of the display device 2 by changingof the orientation of the object image 3 according to the orientationchange of the operation device 1. For example, as illustrated in FIG. 8, the user rotates the operation device 1 so that the marker M1 of theobject image 3 displayed on the display device 2 faces the frontdirection. When the object image 3 rotates in association with therotation of the operation device 1 and the orientation of the marker M1coincides with the front direction, specifically, when the directionperpendicular to the portion (surface) of the object image 3 to whichthe marker M1 is attached matches the direction perpendicular to thedisplay surface of the display 23, the marker detector 213 detects themarker M1.

When the marker detector 213 detects the marker M, the informationpresenter 214 presents the specific information associated with thespecific position. For example, in the example illustrated in FIG. 8 ,when the marker detector 213 detects the marker M1, the informationpresenter 214 displays specific information C1 associated with themarker M1 on the display 23. Similarly, when the marker detector 213detects the marker M2, the information presenter 214 displays specificinformation C2 associated with the marker M2 on the display 23. Further,when the marker detector 213 detects the marker M3, the informationpresenter 214 displays specific information C3 associated with themarker M3 on the display 23. The specific information C1, C2 and C3 are,for example, information such as descriptions and explanations about thetea bowl, and register different information. The specific information Cmay be text information. Further, the specific information C may beimage information such as a photograph or an illustration. Each marker Mand each specific information C are stored in the storage 22 inassociation with each other.

Each time the marker detector 213 detects the marker M, the informationpresenter 214 presents the corresponding specific information C.

Here, when the marker detector 213 detects the marker M, the markerdetector 213 may register information indicating that the marker M hasbeen detected in the storage 22. For example, the marker detector 213registers the detection status indicating “detected” and “undetected”for each marker M. The marker detector 213 does not re-detect thedetected marker M. Therefore, when the marker M is detected once and theinformation presenter 214 presents the corresponding specificinformation C, the specific information C is not re-presented. When themarker detector 213 detects the marker M, the marker detector 213 maydelete the detected marker M from the object image 3. When the operationdevice 1 is placed at a predetermined position, the marker detector 213resets the detection status. According to this configuration, forexample, when the user A appreciates a tea bowl and views all of thespecific information C1, C2 and C3, the specific information C1, C2 andC3 are not re-presented to the user A. Thereafter, when the user Areturns the operation device 1 to the predetermined position, and theuser B starts to appreciate the tea bowl, the specific information C1,C2 and C3 are presented.

Note that the information presenter 214 may output the specificinformation C by voice. For example, when the marker detector 213detects the marker M1, the information presenter 214 causes a speaker(not illustrated) to output the text information of the specificinformation C1 associated with the marker M1 by voice. The speaker maybe provided on the display device 2 or the operation device 1.

Further, the marker M may be displayed only on the object image 3, onboth the object image 3 and the operation device 1, or only on theoperation device 1.

When the operation device 1 moves in a direction away from the displaydevice 2 so that the operation device 1 overlaps a virtual plane S1 at aposition separated from the display device 2 by a predetermined distanceL1, the cross-section display 215 causes the display 23 to display theobject image 3 including a cut plane obtained by cutting the operationdevice at the virtual plane S1. Specifically, first, as illustrated inFIG. 9 , the controller 21 sets the virtual plane S1 at a positionseparated by the predetermined distance L1. The predetermined distanceL1 is preset by an administrator or the like and can be changed asappropriate. The predetermined distance L1 is registered in the storage22. The controller 21 identifies the position of the operation device 1based on the detection result of the position detection sensor 24, anddisplays the cross section of the object image 3 when the operationdevice 1 exceeds the predetermined distance L1. Note that the dottedline of the operation device 1 illustrated in FIG. 9 represents thecutting position by the virtual plane S1. The cross-section display 215is an example of a cross-section display of the present disclosure.

For example, when all the markers M set on the object image 3 aredetected, and all the specific information C is presented, thecontroller 21 enables a cross-section display mode in which thecross-section of the object image 3 can be displayed. While thecross-section display mode is disabled, the controller 21 changes thedisplay magnification of the object image 3 in accordance with themovement of the operation device 1 (see FIGS. 5 and 6 ).

When the cross-section display mode is enabled, as illustrated in FIG.10 , the controller 21 changes the display magnification of the objectimage 3 when the operation device 1 moves within the range of thepredetermined distance L1 from the display device 2, and displays thecross section of the object image 3 when the operation device 1 exceedsthe range of the predetermined distance L1 from the display device 2 sothat the operation device 1 overlaps the virtual plane S1. For example,within the range of the predetermined distance L1, the orientationcontroller 212 displays the object image 3 with a reduced size when theoperation device 1 approaches the display device 2, and displays theobject image 3 with an enlarged size when the operation device 1 movesaway from the display device 2. On the other hand, when the distanceexceeds the range of the predetermined distance L1, the cross-sectiondisplay 215 causes the display 23 to display the object image 3including the cut plane obtained by cutting the operation device 1 atthe virtual plane S1. Further, when the distance exceeds the range ofthe predetermined distance L1, the orientation controller 212 displaysthe object image 3 at a preset magnification. That is, when the distanceexceeds the range of the predetermined distance L1, the orientationcontroller 212 changes the cutting position of the object image 3 inaccordance with the movement of the operation device 1 and does notchange the display magnification.

Further, when the virtual plane S1 cuts a predetermined position of theoperation device 1, the information presenter 214 may present specificinformation C4 associated with the predetermined position. For example,as illustrated in FIG. 11 , when the virtual plane S1 cuts anintermediate position of the operation device 1, the cross-sectiondisplay 215 displays the object image 3 including the cross section cutat the intermediate position, and the information presenter 214 presentsthe specific information C4 associated with the intermediate position.Note that when the virtual plane S1 overlaps the marker M describedabove (see FIG. 7 ), the information presenter 214 may present thespecific information C associated with the marker M. The informationpresenter 214 is an example of an information presenter of the presentdisclosure.

Display Control Processing

Next, display control processing executed in the display control system100 will be described with reference to FIG. 12 . Specifically, in thepresent embodiment, the display control processing is executed by thecontroller 11 of the operation device 1 and the controller 21 of thedisplay device 2. Note that the controllers 11 and 21 may terminate thedisplay control processing in the middle by a predetermined operation ofthe operation device 1 or the display device 2.

Note that the present disclosure can be regarded as a disclosure of adisplay control method (an example of a display control method of thepresent disclosure) for executing one or a plurality of steps includedin the display control processing. Further, one or a plurality of stepsincluded in the display control processing described herein may beappropriately omitted. It should be noted that each step in the displaycontrol processing may be executed in a different order within a rangein which the similar effect is produced. Furthermore, although the casewhere the controllers 11 and 21 execute the respective steps in thedisplay control processing will be described as an example, a displaycontrol method in which a plurality of processors execute the respectivesteps in the display control processing in a distributed manner is alsoconsidered as another embodiment.

Here, the operation device 1 can operate the object image 3 displayed onthe display device 2 by activating a dedicated application.

First, in step S1, the controller 11 of the operation device 1determines whether the orientation of the operation device 1 is set tothe facing orientation (front direction). Specifically, the controller11 activates the dedicated application, and sets the orientation of theoperation device 1 at the position which is the front of the operationdevice 1 to the facing orientation.

The controller 11 sets the facing orientation based on the positioncoordinates and the inclination acquired from the orientation detectionsensor 13. For example, when the operation device 1 is placed at apredetermined position, the controller 11 sets the facing orientation.

When the operation device 1 is set to the facing orientation (S1: Yes),the processing proceeds to step S2. The controller 11 waits until theoperation device 1 is set to the facing orientation (S1: No).

In step S2, the controller 21 of the display device 2 causes the display23 to display the object image 3 in a predetermined directioncorresponding to the facing orientation.

In step S3, the controller 11 of the operation device 1 starts acceptingan operation from the user to the operation device 1. Thus, the user(viewer) can operate the object image 3 using the operation device 1.

In step S4, the controller 11 of the operation device 1 determineswhether the orientation change of the operation device 1 has beendetected. Specifically, the controller 11 detects the presence orabsence of the orientation change of the operation device 1 based on thedetection signal acquired from the orientation detection sensor 13. Forexample, as illustrated in FIG. 1 , when the user rotates the operationdevice 1 in the right direction D1, the controller 11 detects theorientation change (right rotation) of the operation device 1 based onthe detection signal acquired from the orientation detection sensor 13.Further, for example, as illustrated in FIG. 5 , when the user moves theoperation device 1 to the front side, the controller 11 detects theorientation change (position change) of the operation device 1 based onthe detection signal acquired from the orientation detection sensor 13.When the controller 11 detects the orientation change of the operationdevice 1 (S4: Yes), the processing proceeds to step S5. The controller11 waits until the orientation change of the operation device 1 isdetected (S4: No). Step S4 is an example of an orientation detectionstep of the present disclosure.

In step S5, the controller 21 of the display device 2 changes theorientation of the object image 3 displayed on the display device 2according to the orientation change of the operation device 1.Specifically, the controller 11 of the operation device 1 outputs theorientation information including the rotation angle, the tilt angle,the coordinates, the display magnification, and the like acquired fromthe orientation detection sensor 13 to the display device 2. When thecontroller 21 of the display device 2 receives the orientationinformation, the controller 21 of the display device 2 changes therotation angle, the tilt angle, the coordinates, the displaymagnification, and the like of the object image 3 based on theorientation information (see FIGS. 5 and 6 ). Step S5 is an example ofan orientation control step of the present disclosure.

In step S6, the controller 21 of the display device 2 determines whethera marker M set at the specific position of the object image 3 has beendetected. For example, when three markers M1, M2 and M3 are set on theobject image 3 (see FIG. 7 ), the controller 21 determines whether anyone marker M has been detected. When the controller 21 detects a markerM (S6: Yes), the processing proceeds to step S7. When the controller 21does not detect any marker M (S6: No), the processing returns to stepS4. Step S6 is an example of a marker detection step of the presentdisclosure.

In step S7, the controller 21 of the display device 2 determines whetherall the markers M have been detected. For example, the controller 21determines whether all the markers M1, M2 and M3 set on the object image3 have been detected by referring to the detection status of the storage22. When the controller 21 has not detected all the markers M (S7: No),the processing proceeds to step S8. On the other hand, when thecontroller 21 has detected all the markers M (S7: Yes), the processingproceeds to step S9.

In step S8, the controller 21 presents the specific information Cassociated with the detected marker M. For example, when the controller21 detects the marker M1, the controller 21 displays the specificinformation C1 associated with the marker M1 on the display 23 (see FIG.8 ). Step S8 is an example of an information presentation step of thepresent disclosure.

In step S9, the controller 21 presents the specific information Cassociated with the detected marker M as in step S8. In step S9, thecontroller 21 presents the specific information C associated with themarker M detected last among the plural markers M set on the objectimage 3. By repeating steps S4 to S8, all of the specific information C1associated with the marker M1, the specific information C2 associatedwith the marker M2, and the specific information C3 associated with themarker M3 are sequentially presented to the user. When all the markers Mare detected and all the specific information C is presented, theprocessing proceeds to step S10.

In step S10, the controller 21 enables the cross-section display mode.When the cross-section display mode is enabled, the controller 21displays the cross-section of the object image 3 on the display 23 whenthe following conditions are satisfied.

In step S11, the controller 11 of the operation device 1 determineswhether the orientation change of the operation device 1 has beendetected. For example, as illustrated in FIG. 5 , when the user movesthe operation device 1 to the front side, the controller 11 detects theorientation change (position change) of the operation device 1 based onthe detection signal acquired from the orientation detection sensor 13.When the controller 11 detects the orientation change of the operationdevice 1 (S11: Yes), the processing proceeds to step S12. When thecontroller 11 does not detect the orientation change of the operationdevice 1 (S11: No), the processing proceeds to step S15.

In step S12, the controller 21 of the display device 2 determineswhether the operation device 1 exceeds the virtual plane S1 (or whetherthe operation device 1 overlaps the virtual plane S1). Specifically, thecontroller 21 specifies the position of the operation device 1 based onthe detection result of the position detection sensor 24, and determineswhether the tip of the operation device 1 exceeds the predetermineddistance L1 from the display device 2. When the operation device 1 doesnot exceed the virtual plane S1 (S12: No), the processing proceeds tostep S13. On the other hand, when the operation device 1 exceeds thevirtual plane S1 (S12: Yes), the processing proceeds to step S14.

In step S13, the controller 21 of the display device 2 changes theorientation of the object image 3 displayed on the display device 2according to the orientation change of the operation device 1. In thiscase, since the operation device 1 moves to, for example, the rear sideor the front side within the range of the predetermined distance L1, thecontroller 21 displays the object image 3 with a reduced size when theoperation device 1 approaches the display device 2, and displays theobject image 3 with an enlarged size when the operation device 1 movesaway from the display device 2. Thereafter, the processing returns tostep S11. When the operation device 1 changes its orientation within therange of the predetermined distance L1, the processing of steps S11 toS13 is repeated. Step S13 is an example of an orientation control stepof the present disclosure.

On the other hand, in step S14, the controller 21 causes the display 23to display the object image 3 including the cut plane obtained bycutting the operation device 1 at the virtual plane S1 (see FIGS. 9 and11 ). In this case, the controller 21 changes the cutting position todisplay the cross section of the object image 3 without changing thedisplay magnification of the object image 3 in accordance with themovement of the operation device 1. Step S14 is an example of across-section display step of the present disclosure.

In step S15, the controller 21 determines whether the terminationoperation has been received from the user. For example, when the userplaces the operation device 1 at a predetermined place, the controller21 determines that the termination operation has been received. When thecontroller 21 receives the termination operation (S15: Yes), thecontroller 21 ends the display control processing. The controller 21repeats the processing of steps S11 to S14 until the terminationoperation is received (S15: No).

Note that, in the display control processing, when the predeterminedposition of the operation device 1 exceeds the virtual plane S1, thatis, when the virtual plane S1 cuts off the predetermined position of theoperation device 1, the controller 21 may present the specificinformation C4 associated with the predetermined position (see FIG. 11). The predetermined position may be an intermediate position of theoperation device 1 or a position where the marker M (see FIG. 7 ) isprovided. As described above, the controller 11 executes the displaycontrol processing.

Here, the display control system 100 may terminate the display controlprocessing when a predetermined period of time has elapsed from thesetting (step S1) of the facing orientation of the operation device 1.Thus, for example, when a plurality of viewers sequentially view an artobject, each viewer can equally appreciate the art object. Further, thedisplay control system 100 may terminate the display control processingwhen a predetermined period of time has elapsed since all the markers Mhave been detected (step S7). Thus, each viewer can browse all thespecific information C.

Note that the display control processing described above is configuredto enable the cross-section display mode (step S10) on the conditionthat all the markers M are detected (step S7). However, as anotherembodiment, the display control system 100 may enable the cross-sectiondisplay mode when a predetermined operation by the user is received. Forexample, when the user presses a predetermined button (not illustrated)on the operation device 1 or the display device 2, when the user changesthe orientation of the operation device 1 to a predeterminedorientation, when the user gives a voice instruction or the like, thedisplay control system 100 may enable the cross-section display mode.

As described above, the display control system 100 according to thepresent embodiment detects the orientation change of the operationdevice 1 when the user changes the orientation of the operation device1, and changes the orientation of the object image 3 of the operationobject displayed on the display device 2 according to the orientationchange of the operation device 1. Further, the display control system100 detects the marker M set at the specific position of the objectimage 3, and presents the specific information C associated with thedetected marker M to the user (viewer). According to this configuration,it is possible to move the object image 3 in conjunction with themovement of the operation device 1. Furthermore, the user can browseinformation (specific information) relating to the operation object bymoving the operation device 1.

Also, when the operation device 1 moves in the direction away from thedisplay device 2 and overlaps the virtual plane S1 located at apredetermined distance away from the display device 2, the displaycontrol system 100 according to the present embodiment causes thedisplay device 2 to display the object image 3 including the cut planeobtained by cutting the operation device 1 at the virtual plane S1.According to this configuration, the display control system 100 candisplay the cross-section information such as the thickness and theinternal structure of the object image 3 in association with themovement of the operation device 1. Therefore, the user can browse thecross-section information relating to the operation object by moving theoperation device 1.

The present disclosure is not limited to the embodiments describedabove. Other embodiments of the present disclosure will be describedbelow.

In the embodiments described above, for example, when the user holds theoperation device 1 and extends her or his arm, the relative positionbetween her or his face and the operation device 1 is separated, so thatthe size of the operation device 1 as seen from the user becomessmaller. In this case, the object image 3 displayed on the displaydevice 2 is displayed with a reduced size so as to reflect the size ofthe operation device 1 as seen from the user (see FIG. 6 ). Similarly,for example, when the user holds the operation device 1 and folds his orher arm, the relative position between her or his face and the operationdevice 1 approaches each other, so that the size of the operation device1 as seen from the user becomes larger. In this case, the object image 3displayed on the display device 2 is displayed with an enlarged size soas to reflect the size of the operation device 1 as seen from the user(see FIG. 5 ). As described above, when the relative position betweenthe user and the operation device 1 changes by the user bending orstretching of her or his arm at a place (while the user is standing orsitting), the size of the operation device 1 as seen from the user isappropriately reflected on the object image 3.

However, for example, when the user moves (walks) while holding theoperation device 1, it is conceivable that the size of the operationdevice 1 as seen from the user may not be appropriately reflected on theobject image 3. For example, when the user approaches the display device2 on foot while holding the operation device 1, the object image 3 isdisplayed with a reduced size even though the relative position betweenthe user and the operation device 1 does not change and the size of theoperation device 1 as seen from the user does not change. Similarly, forexample, when the user moves away from the display device 2 on footwhile holding the operation device 1, the object image 3 is displayedwith an enlarged size even though the relative position between the userand the operation device 1 does not change and the size of the operationdevice 1 as seen from the user does not change. As described above, whenthe size of the operation device 1 as seen from the user is notappropriately reflected on the object image 3, there arises a problemthat the operability of the operation device 1 and the visibility of theobject image 3 are deteriorated.

Therefore, the display control system 100 according to anotherembodiment includes a configuration capable of solving the above problemin addition to the configuration of the embodiments described above.

Specifically, the position detection sensor 24 detects the position ofthe operation device 1 and the position of the user with respect to thedisplay device 2. Specifically, the position detection sensor 24 detectsthe position (X, Y and Z coordinates) of the operation device 1 and theposition (X, Y and Z coordinates) of the user with the position of theposition detection sensor 24 as a reference (coordinate origin). Theorientation controller 212 calculates the relative position between theoperation device 1 and the display device 2 and the relative positionbetween the operation device 1 and the user, based on the detectionsignal from the position detection sensor 24. For example, asillustrated in FIG. 13 , the orientation controller 212 calculates adistance L2 between the operation device 1 and the display device 2 anda distance L3 between the operation device 1 and the user. Then, theorientation controller 212 changes the orientation of the object image 3based on the distances L2 and L3.

For example, when the user grips the operation device 1 placed at apredetermined place, the orientation setter 111 sets the orientation ofthe operation device 1 to the facing orientation in which the operationdevice 1 faces the user. At this time, the orientation controller 212sets the distance L3 between the operation device 1 and the user as areference distance. Thereafter, for example, when the user extends heror his arm on the place and moves the operation device 1 to the rearside (display device 2 side), the distance L3 becomes larger than thereference distance. When detecting that the distance L3 is changed(increased), the orientation controller 212 displays object image 3 witha reduced size at a reduction ratio corresponding to the amount ofchange. Further, for example, when the user folds her or his arm on theplace and moves the operation device 1 to the front side (user side),the distance L3 becomes smaller than the reference distance. Whendetecting that the distance L3 is changed (decreased), the orientationcontroller 212 displays the object image 3 with an enlarged size at amagnification ratio corresponding to the amount of change.

On the other hand, for example, when the user walks to the rear side(display device 2 side) while holding the operation device 1, thedistance L3 does not substantially change from the reference distance,and only the distance L2 becomes smaller. In this case, the orientationcontroller 212 does not change the display magnification of the objectimage 3 on the condition that the change in the distance L3 is notdetected. Similarly, for example, when the user walks to the front side(in a direction away from the display device 2) while holding theoperation device 1, the distance L3 does not substantially change fromthe reference distance, and only the distance L2 becomes larger. In thiscase, the orientation controller 212 does not change the displaymagnification of the object image 3 on the condition that the change inthe distance L3 is not detected.

According to this configuration, it is possible to appropriately reflectthe size of the operation device 1 as seen from the user on the objectimage 3. Therefore, it is possible to prevent the operability of theoperation device 1 and the visibility of the object image 3 fromdeteriorating. Note that when the controller 21 detects only the changeof the distance L2 without detecting the change of the distance L3, thecontroller 21 may determine that the user is moving while holding theoperation device 1 and display a predetermined message on the display23. For example, the controller 21 may display a message prompting theuser to move the operation device 1 while the user do not move, such as“Please move the replica closer to or away from your body (face)”.

As another embodiment of the present disclosure, the position detectionsensor 24 may be mounted on the operation device 1. For example, asillustrated in FIG. 14 , the operation device 1 may include a camera 15which is an example of the position detection sensor 24. The camera 15is disposed vertically upward so as to be capable of capturing an imageof the ceiling of the room. The operation device 1 detects the position(coordinates) of the operation device 1 based on the captured image(ceiling image) by the camera 15. The operation device 1 outputs adetection signal (coordinate information) to the display device 2. Whenthe display device 2 receives the detection signal, the display device 2detects the orientation change of the operation device 1, and changesthe orientation of the object image 3.

The operation object of the present disclosure is not limited to an artobject, but may be an article in various fields. For example, theoperation object may be an organ, a building, an ornament, or the like.

It is to be understood that the embodiments herein are illustrative andnot restrictive, since the scope of the disclosure is defined by theappended claims rather than by the description preceding them, and allchanges that fall within metes and bounds of the claims, or equivalenceof such metes and bounds thereof are therefore intended to be embracedby the claims.

What is claimed is:
 1. A display control system that changes anorientation of an object image of an operation object displayed on adisplay device in association with an orientation change of an operationdevice, the display control system comprising: a cross-section displaywhich, when the operation device moves in a direction away from thedisplay device and overlaps a virtual plane at a position separated fromthe display device by a predetermined distance, causes the displaydevice to display the object image comprising a cut plane obtained bycutting the operation device at the virtual plane, wherein thecross-section display displays the object image with a reduced size whenthe operation device approaches the display device within a range of thepredetermined distance from the display device, and displays the objectimage with an enlarged size when the operation device moves away fromthe display device within the range of the predetermined distance fromthe display device, and the cross-section display displays the cut planeof the object image without changing a display magnification of theobject image when the operation device is outside the range of thepredetermined distance from the display device.
 2. The display controlsystem according to claim 1, wherein the cross-section display displaysthe object image at a preset magnification when the operation deviceoverlaps the virtual plane.
 3. The display control system according toclaim 1, further comprising: an information presenter that presentsspecific information associated with a predetermined position of theoperation device when the virtual plane cuts the predetermined position.4. The display control system according to claim 3, wherein theinformation presenter causes the display device to display the specificinformation.
 5. The display control system according to claim 3, whereinthe information presenter causes the specific information to be outputby voice.
 6. The display control system according to claim 1, furthercomprising: an orientation controller that changes an orientation of theobject image based on a rotation angle of the operation devicecorresponding to an orientation change of the operation device and aposition of the operation device relative to the display device.
 7. Thedisplay control system according to claim 1, wherein the operationobject is an art object, wherein the object image is a three-dimensionalimage of the art object, and wherein the operation device is a replicaof the art object.
 8. A display control method comprising: changing anorientation of an object image of an operation object displayed on adisplay device in association with an orientation change of an operationdevice; and when the operation device moves in a direction away from thedisplay device and overlaps a virtual plane at a position separated fromthe display device, executing by one or more processors to cause thedisplay device to display the object image comprising a cut planeobtained by cutting the operation device at the virtual plane, whereinthe object image is displayed with a reduced size when the operationdevice approaches the display device within a range of a predetermineddistance from the display device, and the object image is displayed withan enlarged size when the operation device moves away from the displaydevice within the range of the predetermined distance from the displaydevice, and the cut plane of the object image is displayed withoutchanging a display magnification of the object image when the operationdevice is outside the range of the predetermined distance from thedisplay device.